Search results for "Message Passing Interface"
showing 8 items of 8 documents
Big Data in metagenomics: Apache Spark vs MPI.
2020
The progress of next-generation sequencing has lead to the availability of massive data sets used by a wide range of applications in biology and medicine. This has sparked significant interest in using modern Big Data technologies to process this large amount of information in distributed memory clusters of commodity hardware. Several approaches based on solutions such as Apache Hadoop or Apache Spark, have been proposed. These solutions allow developers to focus on the problem while the need to deal with low level details, such as data distribution schemes or communication patterns among processing nodes, can be ignored. However, performance and scalability are also of high importance when…
SAUCE: A Web-Based Automated Assessment Tool for Teaching Parallel Programming
2015
Many curricula for undergraduate studies in computer science provide a lecture on the fundamentals of parallel programming like multi-threaded computation on shared memory architectures using POSIX threads or OpenMP. The complex structure of parallel programs can be challenging, especially for inexperienced students. Thus, there is a latent need for software supporting the learning process. Subsequent lectures may cover more advanced parallelization techniques such as the Message Passing Interface (MPI) and the Compute Unified Device Architecture (CUDA) languages. Unfortunately, the majority of students cannot easily access MPI clusters or modern hardware accelerators in order to effectivel…
VEF Traces: A Framework for Modelling MPI Traffic in Interconnection Network Simulators
2015
Simulation is often used to evaluate the behaviour and measure the performance of computing systems. Specifically, in high-performance interconnection networks, the simulation has been extensively considered to verify the behaviour of the network itself and to evaluate its performance. In this context, network simulation must be fed with network traffic, also referred to as network workload, whose nature has been traditionally synthetic. These workloads can be used for the purpose of driving studies on network performance, but often such workloads are not accurate enough if a realistic evaluation is pursued. For this reason, other non-synthetic workloads have gained popularity over last dec…
A Parallel Implementation of the Tree-Structured Self-Organizing Map
2002
This paper presents how Self-Organizing Maps (SOMs)can be trained efficiently using several, simultaneously executing threads on a shared memory Symmetric MultiProcessing (SMP)computer. The training method is a batch version of the Tree-Structured Self-Organizing Map. We note that SMP type of parallel training is very useful for large data sets obtained from nature, the process industry or large document collections, since we do not encounter similar model size limitations as with hardware SOM implementations.
A Simulation Framework for Evaluating Distributed Reputation Management Systems
2016
In distributed environments, where interactions involve unknown entities, intelligent techniques for estimating agents’ reputation are required. Reputation Management Systems (RMSs) aim to detect malicious behaviors that may affect the integrity of the virtual community. However, these systems are highly dependent of the application domain they address; hence the evaluation of different RMSs in terms of correctness and resistance to security attacks is frequently a tricky task. In this work we present a simulation framework to support researchers in the assessment of a RMS. The simulator is organized in two logic layers where network nodes are mapped to system processes that implement the i…
Network Reconfiguration Suitability for Scientific Applications
2008
This paper analyzes the communication pattern of several scientific applications and how they can make profit of network reconfiguration in order to adapt network topology to the communication needs so that total execution time is reduced. By using an analysis methodology based on real application executions, we study the variation of the required communication bandwidth with time and also the global interprocedural communication patterns. Results show that required bandwidth between each pair of processes does not significantly fluctuates, leading to a constant use of the links and therefore discouraging dynamic reconfigurations of the network during execution time. Nevertheless, the group…
The 1-way on-line coupled atmospheric chemistry model system MECO(n) – Part 2: On-line coupling with the Multi-Model-Driver (MMD)
2012
A new, highly flexible model system for the seamless dynamical down-scaling of meteorological and chemical processes from the global to the meso-γ scale is presented. A global model and a cascade of an arbitrary number of limited-area model instances run concurrently in the same parallel environment, in which the coarser grained instances provide the boundary data for the finer grained instances. Thus, disk-space intensive and time consuming intermediate and pre-processing steps are entirely avoided and the time interpolation errors of common off-line nesting approaches are minimised. More specifically, the regional model COSMO of the German Weather Service (DWD) is nested on-line into the …
EOF-Library: Open-source Elmer FEM and OpenFOAM coupler for electromagnetics and fluid dynamics
2019
EOF-Library is a software that couples Elmer and OpenFOAM simulation packages. It enables efficient internal field interpolation and communication between the finite element and the finite volume frameworks. The coupling of the two packages is based on the Message Passing Interface, which results in low latency, high data bandwidth and parallel scalability. Potential applications are magnetohydrodynamics, convective cooling of electrical devices, industrial plasma physics and microwave heating. In this work we introduce the software and perform interpolation accuracy and parallel scaling tests by sending a known scalar distribution between the two codes. Keywords: Elmer, FEM, OpenFOAM, FVM,…